feat: merge local datasets in the browser - #115
Samuel-Hills wants to merge 3 commits into
Conversation
nicolas-rabault
left a comment
nicolas-rabault
left a comment
There was a problem hiding this comment.
Thanks for the contribution! Please address the issues detailed inline and get the required Quality check passing before merge.
|
|
||
| source_dirs = [episode_media.resolve_dataset_dir(repo_id) for repo_id in source_repo_ids] | ||
| datasets = [ | ||
| LeRobotDataset(repo_id=repo_id, root=source_dir) |
There was a problem hiding this comment.
[P1] Repair interrupted recordings before opening them. A locally recorded dataset can have readable data and meta/info.json but no meta/episodes index. With the pinned LeRobot v0.6.0, this constructor then attempts a Hub lookup instead of reading the local recording, so a never-uploaded dataset cannot merge. I reproduced this with two tiny real datasets after removing one episode index; the complete-dataset case passes. Call the existing repair_local_dataset helper before constructing each source, handle DatasetRepairError, and add an offline regression case. CLAUDE.md's dataset_repair rule requires this before opening recorded datasets.
| )} | ||
| </div> | ||
| <MergeDatasetsDialog | ||
| datasets={datasets} |
There was a problem hiding this comment.
[P2] Carry the current dataset selection into the merge flow. The dialog receives no repoId, starts with an empty selection, and the Merge button is available even when no dataset is selected. I verified the latter in the running browser. This makes users pick both sides again instead of choosing what to merge into the dataset they are browsing. Show the action against a selected local dataset, retain that dataset as a source, and ask only for additional sources; cover opening the dialog from a selected dataset.
| </head> | ||
| <body> | ||
| <div id="root"></div> |
There was a problem hiding this comment.
[P2] Commit the exact generated index.html. Rebuilding this head with Node 22 and the locked dependencies changes this file: the committed version has CRLF line endings and an extra blank line after the root div. The required Quality workflow rejects any dirty dist output, even though the JS and CSS hashes match. Regenerate and commit the index without a subsequent formatting or line-ending conversion, then verify the dist freshness check is clean.
2 participants
Summary
Adds a local-only workflow for merging two or more existing local LeRobot
datasets from LeLab's Dataset browser.
The source datasets remain unchanged. The merged result is written as a new
local/<name>dataset only after the merge completes successfully.Included
POST /datasets/mergeendpointmerge_datasetsintegrationValidation
uv run pytest tests/test_datasets.py -q— 36 passeduv run ruff check ...— passednpm test— 7 passednpm run build— passedCloses #109.